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SUMMARY 



Combat Active Replacement Factors, or CARFs, are 
logistics planning factors currently used by the U.S. Marine 
Corps as estimates of equipment losses in future conflicts. 
Adapted Army replacement factors are currently a prime 
source for CARF values, but verification of these values 
is difficult. 

This report examines two alternate means of estimating 
CARF values for comparative purposes. The first employs 
mean-time-to-loss estimates for various equipment types, 
and several scenario-oriented models are given for mapping 
these values into CARFs. 

Professional military judgement provides another way 
of estimating CARF values, and a procedure is given by 
which the judgement of many experts can be aggregated 
to provide CARF information. A demonstration of the 
procedure is given, employing actual data from twenty- 
three judges who rated "chance of loss" for twenty-one 
equipment types . 
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I. INTRODUCTION 



Combat Active Replacement Factors, or CARFs, are 
logistics planning factors currently used by the U.S. 

Marine Corps. As estimates of equipment losses in future 
conflicts, their values have a significant impact on 
procurement, stockpiling, and plans for shipping requirements. 

Concern over the accuracy of CARF values led to the 
establishment by the Marine Corps in 1983 of a standing board 
for the review of materiel replacement factors. The board's 
specified functions include the review of " . . . replace- 

ment factors for new materiel entering the Marine Corps 
Supply System, existing replacement factors for materiel 
presently in the Marine Corps Supply System, and other 
services' replacement factors . . . where such factors 

would prove useful in confirming the validity of common 
item factors."^ The board's task is a difficult one*, the 
current Marine Corps primary source of CARF values is that 
of adapting Army replacement factors for like items, but 
the Marine Corps lacks available war gaming and simulation 
resources to permit careful verification of replacement 
factors by these means. This report discusses some ways 
of generating comparative CARF values (for verification 
use) using procedures which incorporate military judgement 
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and do not require computerized war games. 

Combat Active Replacement Factors (CARFs) 

I 

The War Reserve Policy Manual of the Marine Corps 
states that a replacement factor is " . . . the estimated 
percentage of equipment in use that will require replacement 
during a given period due to wear-out beyond repair, enemy 
action, abandonment, pilferage, and other causes, except 
catastrophies . . . . The Marine Corps expresses replacement 

factors as quantities required for a 30-day period." 

Further, "The combat active replacement factor will be 
applied for units during those periods when they are actually 
in active combat operations. A force in contact with the 
enemy is considered to be active combat. The combat active 
replacement factor reflects anticipated combat attrition 
of equipment, on a 30-day basis, incident to amphibious 

2 

operations and other combat operations normal to the FMF" . 
Numerical values for CARFs for a wide variety of equipment 
items are listed in the Marine Corps Table of Authorized 

3 

Materiel . 

With Marine Corps support, several studies were con- 
ducted in the early 1980s in an effort to develop a procedure 

4 5 

to generate CARF values. ' These led to a decision to use 
adapted Army replacement factor values.^ In a related effort 
a CARF Determination System Users Manual was prepared."^ 
Concern over verification has remained, and in 1985 a number 
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of existing war games were surveyed with regard to their 
usefulness to the Marine Corps for CARF generation and 
verification.^ , 

This Report 

This report looks at several ways of employing military 
judgement to estimate CARFs for verification purposes. 
Although CARF values could be subjectively estimated dir- 
ectly, one alternative is to employ estimates of the average 
combat survival time of an item, and in the next section 
we will look at ways in which Mean Time Until Loss (MTTL) 
values may be converted into CARF values. 

Another way to acquire professional military judgement 
about CARF values is to ask that various equipment items 
simply be ranked with regard to their CARF values. 

Section III of this report explains how such rankings by 
many judges may be combined to yield CARF values for the 
various items. A demonstration of this method is then 
given, scaling chance of loss for 21 functional areas 
of equipment. The report concludes with appendices 
providing some cautions regarding CARF generation and 
use, and the mathematical derivation of the scaling method 
which uses rankings. 



:il. CARF GEl^ERATION AND VERIFICATION 
USING MTTL ESTIMATES 



There are alternatives to the use of computerized war 
games and simulation to obtain values for CARFs. It is 
possible to produce estimates directly, perhaps using 
professional military judgement and experience. In some 
cases CARF values so generated may be preferred to those 
obtained from combat models, since there may be more clarity 
about what the number was based upon, and what considerations 
went into its estimation. 

A variation on estimating CARFs directly is to use 
professional judgement to estimate a related measure, the 
mean time until loss, or MTTL. The MTTL is the average 
time one would expect an item to survive in the combat 
environment (the analogy to MTBF is appropriate) . For some 
items and some individuals asked for judgements, the MTTL 
may be a more natural value to estimate directly than a 
replacement factor. 

Given an estimate of the item's MTTL, however, the 
relationship between MTTL and CARF is not direct. What is 
needed are procedures by which an MTTL estimate for an item 
could be used to produce a CARF value for the item. Such 
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procedures necessarily depend upon the process by which the 
losses occur. We shall look at several kinds of scenarios 
by which losses could occur in combat, and then develop 

I 

relationships permitting a CARF value to be obtained from 
an MTTL value. These loss processes include, for a specific 
item: 

1. Cases where all items of that type are vulnerable 
(subject to possible loss) initially and throughout 
the combat period, at the same rate. 

2. Cases where subsets of the items in use have 
different loss rates. 

3. Cases where the rate of loss changes at some 
designated time during the combat period. This 
permits representation of an amphibious phase, 
or a force buildup. 

4. Cases where one item is vulnerable initially and 
throughout the combat period at the same rate, but 
its replacements are not vulnerable until put into 
use. 

Development of MTTL-CARP relationships for these 
loss processes involves characterization of the probabilistic 
factors by which the loss of an item occurs. 



Loss Process Characteristics 

We will look at a loss process where the event that an 
item is lost is independent of the loss of any other on-line 
item of the same kind, and where the chance of a surviving 
item being lost on the next day is independent of how many 
days it has already been in combat. This means that the 
individual item' s loss rate may be considered constant 
over the period of time we are examining. 
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For constant loss rates and independent losses, the 
probability distribution for the time an item survives in 

9 

combat will be the exponential distribution. Let t be 

* 

the combat survival time for a specific item, and let R 
be that item's loss rate. (The units of R are items per 
day.) The density function for the time t until the item 
is lost is 

f(t) = Re“^^ , t>0, R^O . 

From this we may immediately obtain the probability that 
the item is lost on or before time T, F(T), as 

— TR 

F(T) = Pr(t<T) = 1 - e . 

We also have 1 - F(T), the probability that the item 
is still surviving just after time T, as 

1 - F(T) = Pr(t>T) = e"”^^ . 

Since the expected value or average value of an 
exponentially distributed random variable is the reciprocal 
of its parameter, we now have the mean time until loss 
MTTL as 

MTTL = 1/R 

Accordingly, the chance that the item is not lost during 
30 days of combat is 

Pr(Not lost during first 30 days) = g'^O/MTTL^ 
or more generally, 

Pr(Not lost during first D days) = e“l^/^TTL^ 



with this characterization of the general loss process 
established, we can now look at various loss scenarios. 
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CARFS When All Items are Vulnerable Initially with the Same 
Loss Rate for 30 Days 

We look first at the case where all items are committed 
and vulnerable at the same loss rate throughout the 30-day 
period. The chance that an item is lost during the 30 days 
is 



If n items are the initial in-use amount, committed with 
independent losses, then the nvrmber of items x that would 
be lost during the 30-day period will be binomially dist- 
ributed, and we have 



The average number (out of n) lost in 30 days is simply the 
mean of the binomial distribution, or 



^-30/MTTL 



Pr 




x=0,l, ...,n. 



( 2 ) 



Ave number lost = n(l - e 



-30/MTTL 



From this we may readily obtain a CARF value: 



CARF 



(Ave number lost in 30 days) (100) 
(in-use amount) 



or 



CARF = (100) 



(3) 
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Equation (3) permits a CAFIF to be computed from an MTTL- 
estimate for the case where each item in the in-use amount 

is committed initially and at the same loss rate, and where 

« 

losses are independent. 

Another use of (3) in the verification of an existing 
CARF value would ask if the existing CARF value would yield 
an MTTL that seemed reasonable. Here, we could use 

MTTL = -30/ln(l-CARF/100) . (4) 

The association between CARF and MTTL for this case 
is illustrated numerically by the values in Table 1. 



TABLE 1. Some Associated CARF 
and MTTL Values for the Case of 
All Items Vulnerable Initially, 
and at the Same Loss Rate for 
30 Days. 



MTTL: Ave. Nvimber 

of Days Before Loss CARF, % 



20 days 


78 % 


30 


63 


40 


53 


50 


45 


100 


26 


200 


14 


350 


8 


500 


6 
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CARFs Where Not All Items Have the Same Loss Rate 

If all items are init^lly vulnerable, but with 
different (constant) loss rates for the 30 day period, 
then CARF generation from the MTTL values is an extension 
of the case where all items have the same loss rate. 

For a general formula, let proportions P 2 'P 2 ' 
of the in-use amount n have mean times until loss 
MTTLj^, MTTL 2 , ... , l-lMMLj,. The average number lost in 

30 days for any subgroup i is 

,, -30/MTTL., 

p^n(l - e ' 1 ) , 

and we have 



CARF = 


f fp.a - 


1 (100) 




1 i=l i 


f 



As an example of the use of (5) , suppose half of the 
in-use amount have MTTL of 200 days, one-quarter have MTTL 
of 100 days, and the remaining quarter have MTTL of 30 days. 
Then from (5) , 

CARF = 100(i5(l - + %{1 - + h U - ) 

= 100(J5(0.14) + hiO.26) + !j(0.63)) = 29.25 % . 

This is, of course, simply a weighted average of CARF 



values from Table 1. 
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CARFs Where All Items are Vulnerable Initially and There 
is a Change in the 'Loss Rate During the 30-Day Period 



It may sometimes be of interest to construct a CARF 
for a situation where there is a change in the combat 
scenario during the 30-day period. One immediate example 
of this is the case where the first portion of the 30 days 
is an amphibious operation, and the loss rate might 
subsequently change. For this type of situation, we let 
1/MTTLj^ be the loss rate for the first D days where 
items are lost, and I/MTTL 2 be the loss rate for the 
remaining (30 - D) days where X 2 items are lost. Here, 
total losses for the 30-day period are x^^+ X 2 , but the 
problem is complicated by the fact that the initial in-use 
amount for the second period is n - x^^, i.e., it depends 
upon losses during the first period. 

In particular, we have the average total losses for 
the 30-day period as 

n 






6 ) 



xi=0 

Substituting (2) and (3) into (6) yields 



e[xi+Xj] = e[xJ -< 




- (30-D)/MTTL. 
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This simply gives us 



e[xi+X 2 ] = n 
The CART would be 



^ _ g-(D/MTTL^ + (30-D) /MTTL 2 ) 



CART = 



_ g-(D/MTTLj^ + (30-D) /MTTL 2 ) 



100 . (7) 



More Complex Scenarios 

The methods of the two preceding sections may be applied 
for the purpose of generating CARFS in other, more complex 
scenarios. A single example would be that of a reserve force 
committed at some time during the 30 days. The total initial 
number of items might be divided between an on-line force 
and a reserve force, with different MTTL values. Commitment 
of the reserve at some time D during the 30-day period would 
decrease its MTTL, and possibly increase the MTTL of the 
initial on-line force. Equations developed in the preceding 
sections offer the tools to construct CARFs for this 
scenario, and for others with more complexity. 
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CARFS Where an On-line Item is Replaced by a Heretofore 
Invulnerable Item 



The expressions for generating CARFs in the previous 
sections have all been based upon a situation where all 
items were initially vulnerable (although loss rates 
could have been zero) . A different loss process which we 
now examine would place one item on line, and structure 
its (possibly repeated) replacement with heretofore 
invulnerable items. 

If the supply of reserve items is very large, or large 
enough so that the chance of its exhaustion is negligible, 
we can structure this case by noting that the loss process 
is simply a sequence of exponentially distributed time 
intervals over a 30 day period. If we ignore boundary 
conditions, the niimber of such intervals would be a 
Poisson distributed random variable with a mean of 
30/MTTL.^ The CARF would be 



CARF 



30/MTTL 
in-use amount 











100 


/ 







( 8 ) 



For CARF verification we could ask if 



(30) (100) 

MTTL = (9) 

(CARF) (in-use amount) 



was a reasonable number. 
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Equations (8) and (9) may be a reasonable approximation 
for an item with a low CARF value. If the chance of running 
out of replacements is not negligible , however, then (8) will 
yield an overstated CARF value. We can correct for this 
(at the cost of simplicity) as follows. Let n be the initial 
amount of an item, and x be the losses in 30 days. Then, 
with the Poisson probability distribution we can write 



(3 0/MTTL) 



X I 



, x=0 , 1 , . . . , n- 1 



f (x) = < 



( 9 : 



( 30/MTTL) ^e 



X 1 



X = n 



x=n 



as the loss distribution. Taking expected values, this 



yields a CARF expression 




( 10 ) 



which is easily programmed for numerical solutions. 
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Remarks 

The methods given in this section provide a way of 

I 

generating CARF values without resort to war games or to 
simulation. Basic to this method is the estimation of 
values for MTTL, the mean time until a loss, and the method 
will be most applicable to situations where the MTTL can be 
estimated with more confidence than the needed CARF can 
be estimated. 

The process assumptions leading to the use of the 
exponential distribution for time until loss are analogous 
to the constant failure rate assumption which is so widely 
used in reliability theory. In the reliability context 
this assumption is widely discussed in that it portrays 
chance failures, but not those stemming from increased age 
of the item, or wearout. Constant failure rates, however, 
are widely used and are essentially mandated by DOD 
needs for MTBF, rather than mission, reliability spec- 
ifications. For many items in our 30-day combat scenario, 
the effects of longevity on item loss may be negligible, 
and the constant loss rate structure as we have employed 
it not a source of excessive concern. 

In the next section we will look in more detail at 
a procedure by which professional judgement could be used 
collectively to estimate or verify CARFs or MTTLs. 



III. CARF GENERATION AND VERIFICATION 
USING MILITARY JUDGEMENT 



A frequent procedure for eliciting expert opinion 
from knowledgeable individuals is that of asking them to 
do some form of ordinal rating of various items with regard 
to some property. Judges usually can provide ordinal ratings 
or ranking with much greater confidence than they can state 
numerical values. For example, from professional expertise 
one should be able to state which of two items should have 
the greater CARF value, and do this with more assurance than 
to state estimated CARF values for each of the two items. 

The method given on the following pages allows ordinal 
information furnished by judges to be combined with a model 
of judge behavior to obtain an interval scale. Thus while 
judges may be asked only to rank items in terms, say, of 
their loss rates, the collective inputs of many judges permit 

1 

and interval scale to be inferred without arbitrary scoring. 
Then, specification of CARF values for two of the items 

should yield the needed ratio-scaled CARF estimates for 

all the items being considered. 

Although ordinal ratings by judges may arise in many 
diverse situations, in too may cases such data is gathered 
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without a clear notion of how it will be processed in order 
to have a resulting scale that collectively represents 
the inputs of the judges. 

A commonly used (but not recommended) approach to 
dealing with a set of lists of rankings is to award arbit- 
rarily chosen numerical values for various ranking positions, 
and then to compute an average score for each item. For 
example, if there are ten items to be scaled, one might 
award nine points if a judge ranks an item first, eight 
points if he ranks it second, and so on. The procedure 
imposes an interval scale interpretation upon the ordinal 
data furnished by the judges, and in most cases is very 
difficult to justify. Further, from a sensitivity analysis 
point of view, it may be shown that the results, and even 
their rank order, often depend dramatically upon the 
scoring system used.^^ 

On the following pages we will first talk about 
various ways to obtain expert judgements from knowledgeable 
persons. Then, we will present a step-by-step procedure 
to convert the rankings to an interval, and then to a ratio 
scale permitting CAKF estimation. The mathematics supporting 
the procedure are presented in Appendix B . 
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Obtaining Information From Judges 

There are several diff,erent ways of obtaining rankings 
from judges, and when selecting a particular approach one 
would want to consider both the number of items for which 
values are sought, and the number of judges who are to be 
queried. Tradeoffs may have to be made between the effort 
that will be required of a judge and the amount of confid- 
ence one wishes to have in the resulting scale values. 

We begin by discussing various ways in which judges 
might record a full ranking of the items. Then, we will 
describe a paired comparisons approach used by psychologists 
and others. 



Ranking all the Items 

A direct scheme for querying a judge is to present 
him with a set of items together with instructions that 
he is to rank them in terms of their CARF values. (Alter- 
natively, we could ask him to do the ranking in terras of 
MTTL, or loss rate, or chance of loss.) As part of these 
instructions, he should be encouraged to rearrange and 
adjust his ordering until he is satisfied with it. 

One way for the judge to express his rankings is to 
write the names of the items in a ranked list. The list 
gives him and others an easily inspected display of his 
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work, and thereby facilitates any subsequent rearrangement 
he may wish to do. Rather than writing the names of the 
items, the judge could write symbols (numbers or letters) 
representing the items. (For example, he would place item 
Number 4 first, item Number 1 second, and so on.) This is 
easy to do, but his opportunity for easy inspection is 
hindered by the coding. Further, unless instructions 
are very clear, there can be ambiguity about the results 
in that if a 3 is listed first, he might have meant that 
item Number 1 was third, rather than item Number 3 first. 

It is generally felt better to ask the judge to use letter 
symbols or abbreviations of the item names. Also, it is 
essential to make clear which goes at the top of the list: 
the item with the highest chance of loss, or the item with 
the lowest chance of loss. 

A different procedure asks the judge to v/ork directly 
with the given list of items, indicating on that list which 
is ranked first, second, and so on. This approach makes it 
very difficult for the judge to inspect his work to see if 
he is satisfied with the rankings he has given. 

A very efficient scheme tending to encourage rearrange- 
ment and to eliminate logic errors is to have the items 
''^^^bten on cards — — judges spread out the cards and arrange 
them to their satisfaction. 

Occasionally, a judge will feel that he cannot discrim— 
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inate between two items in terras of their chance of loss. 
Also, there will be cases where a judge wishes not to rank 
an item because he feels he does not know enough about it. 
Although one will wish to encourage judges to rank all 
items to the extent that they are able, it is undesirable 
to require them to furnish information which they do not 
feel they can express. In the method given in this report, 
information from judges who omit items or who list ties 
may be used completely. 

The Method of Paired Comparisons 

A method of obtaining ordinal information from judges 
which has received considerable attention in the literature 
of Psychology is the Method of Paired Comparisons ^ ^ , 16 

Here a judge is presented with pairs of items and is asked 
to indicate (mark, circle, etc.) which item in the pair 
has the higher loss rate. This data may be incorporated 
directly into the scaling procedure that will be presented 
here. If there are n items to be rated, this involves 
n(n-l)/2 comparisons, so that if there are 20 items, the 
judge would look at 190 pairs. Aggregate rankings, which 
are inherent in the total ranking approach, will not be 
possible. Ties are treated as omitted comparisons. 

Apart from the large number of comparisons that 
may have to be made, another problem with paired comparisons 
is that of insuring logical consistency. A judge might 
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rank A over B, B over C, and then C over A. It may be 
argued that such an inconsistency is properly part of the 
judge's expression of his feelings, and should be accepted. 
The practical problem that remains is the judge's efforts 
to try to avoid this, even if asked not to. This adds to 
the busywork of his task, and may influence the spontaneity 
of his ratings. 



Procedure for Obtaining Scale Values 

1. Data from the judges' ratings (ordinal) may be 

tallied in an f^j array, where f^^ is the nxamber of judges 

who ranked item j as possessing a greater, say, loss rate 

than item i. When this is done, f . + f. will be the 

ab ba 

number of judges who compared item a with item b. 

2. The p^j array may be computed from the f^^ array 
as the proportion of judges comparing i and j who rated 
item j as having a greater loss rate than item i. Here, 






ID 



f . . + f . . 
ID Di 



( 11 ) 



Note that t ~ 1* the diagonal of the Pj^j array. 



we set Pj^j_= 0.5 



3. Using a table of the standard normal distribution 
F(z), we prepare an array where z^^ is the standard 

normal value coiresponding to p^ ^ , 



or 
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J-D 



1 ) 

f ( ^ ) d •/ = 



F(z, 



We omit (leave blank) entries corresponding 

or p. .< 0.02. The z. . array will have zeros on 
11 11 

diagonal, and if a and b are items, z^j^ = - 

4. If the z^j array has no missing values, 
column averages as the preliminary scale values 
n items. These column averages 



S . ^ i^l 



I , 



1 



. j=l,2,...,n 



to p . . > 0.98, 
11 

the main 

we use the 
for the 



( 12 ) 



are the loss rate values for the n items on an interval 
scale, that is, a scale with arbitrary origin and unit. 

5. If the z^j array has missing values, the procedure 
is as follows. We compute column averages for those 
columns which are complete, and use these column averages 
as scale values for those items. For incomplete columns, 
we write a set of linear equations of the form 



n . S . - } S . = y . 

1 1 • r 1 . 1 1 > 

K l . 1 f N . 






(13) 



where ^ denotes the set of n^ elements in column j of the 



z^j array. Substituting the scale values already determined 
by averaging values in the complete columns, we solve the 



set of simultaneous equations to obtain the remaining 
scale values. 
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the foregoing are on an interval scale, and may be changed 
to other interval scales by changing origin and unit 
via any linear transformation of the form 



To obtain the final scale values we need to add information 
which will specify the proper unit and origin. This can 
be done by estimating final scale values for exactly two 
of the items, call them g and h. (For example, we might 



obtained by transforming all the preliminary scale values 



The values found by (15) will be the end result of the judges 
ranking of the items. If items were ranked in terms of 
their CARF values, then after estimating CARF values for 
two items g and h. Equation (15) yields the estimated 
CARF values for all of the items. One useful way to 
employ this method is to add to the list of items which 
are to be scaled to obtain CARF values two additional 
items for which CARF values exist and are felt to be 



s'. = A + BS . 
3 3 



B>0 



(14) 






(15) 



satisfactory. Judges rank all the items, and at the end 
of the data analysis the two items with knownCARFs are 
used to position the scale. 

Remarks 

The procedure presented here employs collective 
military judgement to generate estimates of factors 
describing loss properties of various types of equipment 
Input data is simply in the form of ranked lists, while 
the final values are essentially ratio scaled when the 
procedure is used to estimate such factors as loss rates 
MTTL values, or CARFs . 

The method has the disadvantage that it is based 
upon disagreement among judges; if judges are in close 
agreement in their ranked lists, many items will be 
unsealed except in an ordinal sense. A related concern 
is that a reasonably large number of judges is needed. 
One cannot, for example, expect much success if only 
nine or ten judges are available. 

A demonstration of this procedure employing real 
judges and data is given in the next section, where we 
look at obtaining values for chance of loss for 21 



functional areas of equipment. 



IV. DEMONSTRA.TION OF THE METHOD: 



CHANCE OF LOSS FOR TWENTY-ONE 
FUNCTIONAL AREAS 

As a demonstration of the use of military judgement to 
estimate chance of loss of various kinds of equipment, 
twenty-three knowledgeable judges were asked to rank 
twenty-one types of equipment in terms of "chance of loss 
of individual items". The equipment types were identified 
by management function area codes, and the list of items is 
given in Table 2. 

Raw Data 

A replica of the ranking form is shown in Table 3. 

Judges wrote area code numbers on their lists, rather than 
equipment names . Of the twenty-three judges, only five rated 
all the items: most omitted two or three, although one 
ranked only five items. 

The instructions said nothing about writing ties on the 
same line, but seven of the judges did this. Since the 
scaling procedure handles ties and omitted items, all the 
data furnished by the judges was used. 

It is a characteristic of the scaling method that if 
all judges rank an item first (or all rank it last) , then 
that item cannot be scaled except in an ordinal sense. 
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TABLE 2 . Table of Management 
Functional Area Codes 



FA Code 



Description 



10 

13 

14 
16 
17 

19 

20 
21 
23 
26 

29 

30 
35 

40 

41 

42 

43 
46 

49 

50 
11 



Radios 

Air Command/Control Equipment 
Air Support Radar/IFF Equipment 
Electronic Equipment 
Ground Support Radar 
Intelligence/Surveillance Equipment 
Generators 

Environmental Control Equipment 
Earth Moving Equipment 
Materials Handling Equipment 
Engineer Support 
Trucks 

Towed Motor Transport Equipment 
Tanks 

Amphibious Assault Vehicles 

Light Armored Vehicles 

Artillery 

Infantry Weapons 

Missile Systems 

High Density/Low Deadline 

Communications Support Equipment 



TABLE 3- Demonstration Questionaire 



Please rank the listed functional HIGHEST CHANCE 

OF LOSS 

areas in terms of the chance of 
loss of individual items in the 
functional area. For example, 
if you think that Towed Motor 
Transport Equipment has the high- 
est chance of loss, you would 
write the name, or the code '35' 
on the top line. Omit any 
functional areas you are unable 
to rank. 



LOWEST CHANCE OF LOSS 
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This was the case with Item 50, High Density/Low Density, 

It was ranked by fewer judges (nine) than any other item, 
and most placed it last on, their lists. No two judges ranked 
it above any other specific item. As a result, it was 
dropped from the analysis, to reappear in the final scale 
where it would be declared as having a lower chance of 
loss than the lowest item scaled. 

Of the remaining twenty items, judge's omissions 
were scattered over the items. The exception was Item 11, 
Communication Support Equipment, which was ranked by only 
ten of the twenty-three judges. Since frequencies were good 
on the other items. Item 11 was allowed to remain in the 
analysis . 

Obtaining the Interval Scale 

Scaling twenty items simultaneously is a larger than 
typical problem, and it was necessary to extract paired 
information from the lists for the f^^ array by using a 
computer, since the task was too large to be done by 
manual methods. When cells with p^^j values which were too 
large or too small were deleted, there remained six com- 
plete columns out of the original twenty. This meant that 
six items could have their scale values computed easily, and 
then a set of 14 simultaneous linear equations would have 
to be solved to obtain the remaining scale values. 
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An easier approach was taken. The data was divided 
into two scaling problems, each with eleven items so that 
there were two in common. The sets of items were: 

#1: 10,13,14,16,17,19,20,40,41,42,43; and 
#2: 10,11,16,21,23,26,29,30,35,46,49. 

Scales for the items in each problem were found using the 
procedures described in Section III of this report, and 
then the two scales were merged by a linear transformation 
based upon the two common items. 

This resulted in an interval scale for chance of loss 
for the 20 equipment items; the scale is shown in Figure 1. 
Note that no numbers appear on the scale since they would 
be misleading: scale unit and origin are, at this point, 
totally arbitrary. 

Reference Points 

Two of the items were assigned chance of loss values: 
Trucks at 0.0480, and Tanks at 0.3902. These values were 
specified independently of the ranking information. 

Interval scale values for the entire set of equipment 
areas as shown in Table 4 were transformed to a scale 
that would include the two reference points. This was 
done via Equation (15) , which in this case was 



L . 
3 



-0.4658 + 0. OIS . 

3 
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FIGURE 1. Interval Scale for 
Chance of Loss Data 



41 Amphib. Vehicles 
40 Tanks 

13 Air Command/Control Equip. 

42 Light Armored Vehicles 

14 Air Support Radar 

43 Artillery 



30 Trucks 
10 Radios 



49 Missile 
Systems 




17 Ground Support Radar 
16 Electronic Equipment 

11 Commo Support Equip. 

35 Towed Motor Trans. Eq. 
29 Engineer Support 
46 Infantry Weapons 
19 Intell/Surveillance Eq. 
23 Earthmoving Equipment 
26 Materials Handling Eq. 



21 Environ. Control Eq. 

20 Generators 

50 High Density/Low Deadline 
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TABLE 4. Scaling Results for 
Twenty-one Equipment Functional 
Areas, Chance of Loss 



Chance of Loss, 
based upon: 



FA Code 


Interval Scale 
High 90, Low 10 


L 30 = 0.048 
L 40 = 0-390 


10 


48.9 


0.0221 


11 


41.1 


-0.0558 


13 


82.6 


0.3583 


14 


67.1 


0.2036 


16 


44.1 


-0.0258 


17 


44.5 


-0.0218 


19 


34.6 


-0.1206 


20 


10.0 


-0.3660 


21 


21.9 


-0.2473 


23 


33.9 


-0.1276 


26 


29.3 


-0.1735 


29 


37.3 


-0.0936 


30 


51.2 


0 . 0480* 


35 


37.2 


-0.0947 


40 


85.8 


0.3902* 


41 


90.0 


0.4321 


42 


75.5 


0.2874 


43 


63.7 


0.1697 


46 


36.9 


-0.0976 


49 


47.1 


0.0041 


50 


. Below 10 


Below -0. 
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The resulting values are shown in Table 4. 

The presence of negative values in the right-hand 
colxomn of Table 4 deserves comment, since the terms "chance 

I 

of loss" could be thought of a denoting a probability, and 
thus a non-negative value. In fact, there is nothing in 
this method to restrict values from being negative. Had 
the scale results all been positive, they probably would 
have been interpreted as probabilities without thought 
of justification. The presence of negative values serves 
as a useful reminder that "chance of loss" need not 
necessarily be a probability. 

On the other hand, if one insists that probability 
values should result, then one can ask where errors might 
lie. The basic question when attempting to confirm a 
value by generating it from a different source is what to do 
when the two values differ. (Agreement supports the existing 
value.) In this case, if we insist that the chance of loss 
values should lie between zero and 1.0, then judge and 
reference values are in disagreement: one, the other, or 
both are in error to a conspicuous extent. 

We note that the judges rated 14 of the 21 items (more 
than half) as having a chance of loss lower than Trucks, 
and that the reference value for truck, 0.048, was, as a 
probability, already a low value. Had the reference value 
for Trucks been higher, positive scale values would have 



resulted . 
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One way to investigate this would be to repeat the 
study, possibly with a fewer number of items, but retaining 

#30, Trucks, and #40, Tanks. Then, to the list of items that 

« 

judges would be asked to rank we would add two, described 
as "A functional area whose chance of loss is 0.048" and 
"A functional area whose chance of loss is 0.39". If these 
values are consistent with military judgement, about half 
the judges would rank trucks above 0.048, and about half 
below. Similarly for tanks, about half would rank tanks 
above 0.39, and about half below. 

Finally 

This demonstration served as a first effort to use 
collective rankings to value factors such as chance of 
loss. The data provided a judgement-based interval 
scale for twenty types of equipment, and disagreed in a 
ratio scale sense with two reference points. 

It is hoped that the work presented here will be 
useful to those interested in applying judgemental methods 
to the verification of CARFs and similar measures. 



APPENDIX A 



SOME CAUTIONS REGARDING CARE GENERATION AND USE 



There are many ways to estimate a CARE value, but in 
essence the estimate of losses upon which the CARE is based 
will be in one of two basic forms. Either the entire loss 
ratio will be estimated (as a percentage, proportion, or 
a fraction) , or the number of items that will be lost will 
be estimated. 

In the case where the loss ratio is estimated, the 
denominator of that ratio is left implied. An example would 
be an estimate using military judgement, "I think we'd 
lose 10% of that kind of equipment". Here it is not clear 
what quantity the 10% refers to. It is tempting to assume 
that the reference is to the T/E allowance for forces in 
contact with the enemy, consistent with CARE definition 
and interpretion . A person providing the estimate who 
has worked with values reflecting the quantity of item in 
the EME, however, might be thinking of that reference when 
he states 10%, while a company commander might be using a 
different reference. Generally, when estimates are given 
directly as loss ratios, there is no guarantee that the 
reference will be the desired one unless this is made very 
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clear to the individual making the estimate. Errors can 
be substantial. Also, this kind of estimate probably 

I 

requires knowledge of the in-use amount. 

An alternative form of loss estimate upon which a 
CARF might be based is the number of items lost and requiring 
replacement in a 30-day period. This is the form of estimate 
that would result when a computerized combat model is used 
to produce a CARF value. A direct statement of losses could 
also come from military judgement, e.g., "I think we'd lose 
two" . Here the stated estimate is independent of the 
reference value, at least explicitly, and thus the denom- 
inator for the CARF is left to be specified separately. 

Figure 2 shows the anatomy of a CARF, detailing the 
components involved in producing a number which satisfies 
the definition of a replacement factor, viz., the expected 
percentage of equipment that will require replacement 
. . . for a 30-day period. Our interest here is not only 
the numerator and denominator, but also the fact that the 
definition requires a CARF to be a percentage. Real and 
potential difficulties with both of these characteristics 
of a CARF are identified and discussed in this appendix. 
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The Denominator: The "In-Use" Amount 

A CARF is a percentage, and as such is essentially 
dimensionless. When used for planning and other purposes. 



The CARF estimation problem 
lies primarily in obtaining 
realistic values for losses. 



CARF = 



- 




m •• 


Losses : the number of items 






that will require replacement 






during a 30-day period 




100 


Number of items in use 











Failure to place correct values here 
can lead to CARFs which are substan- 
tially in error, even though losses 
are well estimated. 



FIGURE 2. 



The Anatomy of a CARF 
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its most frequent application will be that of multiplying 
a quantity of equipment to establish the quantity that will 
need replacement. Accordingly, it is crucial if errors 
are to be avoided that the quantity the CARF multiplies 
be of the same substance as the denominator used to determine 
the CARF value. 

There are several ways to interpret in-use amount, 

and as suggested above, clarification might usefully be 

based upon CARF application. This appears to have been done 

2 

as follows, quoting from the War Reserve Policy Manual . 

1. "The combat active replacement factor will be 
applied for units during those periods when they 
are actually in active combat operations. A force 
in contact with the enemy is considered to be in 
active combat." (pg 4-3, 4-4) 

2. "When applied to the density of equipment, the 
combat active replacement factor determines the 
amount of equipment which must be replaced each 
month to maintain the full T/E allowance ." (pg 4-4) 

3. " . . . (CARFs) multiplied by the equipment 

density or unit Table of Equipment (T/E) 
allowance." (pg 12-5) 

As has been said, as long as the denominator used in 
specifying the CARF and the quantity the CARF multiplies 
are of the same nature, there will be no difficulty. 

However, we will have errors that may be substantial when 
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a CARF is used to multiply, say, the number of items in 
the FMF. 

Another area where there may be difficulty with the 

I 

CARF denominator occurs when a CARF is based upon an Army 

replacement factor, called a WARF. The Army's 

"Wartime Replacement Factor (WARF) is the 
average daily catastrophic (nonrepairable) 
item loss rate expressed as a percent of 
the average authorized item strength in 
the combat theater". 

To let CARF=(WARF) (30) means (assuming all other aspects 
of Army and Marine operations to be similar) that "average 
authorized item strength in the combat theater" is the same 
concept as the Marine's "in-use amount". 

Percentage or Proportion ? 

Current numerical values for CARFs for various items 

3 

are listed in the Table of Authorized Materie l (TAM) . 
Consistent with definitions given elsewhere, the TAM defines 

replacement factors as percentages and then lists the 

values for "replacement factors for 30 days" beginning on 

page 26-1. However, the values that are listed are in fact 

proportions, 100 times smaller than the replacement factors 

described. It is apparently true that Supply Officers have 

been trained to interpret these values as proportions. 

However, others seeking CARF values will probably go to the 

TAM as the basic and most available source. The TAM says 

clearly that they are percentages, and thus for unknowing 



38 

users, the values they extract and use are actually under- 
stating losses by a factor of 100. 



APPENDIX B 



DEVELOPMENT OF THE RANKING METHOD 



The procedures in Section II of this report employed a ranking 
method by which experts (called judges) expressed their feelings about 
loss rates or CARF values (scale values or amount of the parameter) 
associated with various items of equipment (instances)) by ranking 
the items. The collective rankings of the judges were then used 
to produce interval scaled values for the amount of the property 
possessed by each of the instances. 



There are several ways to approach scale development from ordinal 



data. Models vary, depending upon what assumptions are made. We shall 
present what is probably the most widely used approach. We assume 
the following: 

a. 

scale value of instance j , but is able to rank instances in 
accordance with his feelings. 



A judge cannot directly express his feelings x^ about the 



b. 



Over the population of judges, 
random variable. 



x . 



3 



is a normally distributed 



c. All instances possess the same variance* for x, so that 

2 2 
0 . = o . 
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d. The correlation coefficient for x between any pair of 



instances is the same, so that p. . = p. 

13 



From these assumptions we may deduce the following. Let i and j be tw 
instances. A judge's feeling about the amount of the property possess 



by instance i is a normally distributed random variable with mean 



and variance o , and a judge's feelings about the amount of the pr 



possessed by instance j is normally distributed random variable Xj wit 

2 

mean S . and variance a . Since the difference between two normally 

D • 



distributed random variables is itself a normally distributed random 



2 2 2 

variable, (x^^-Xj) is normal with mean and variance a + o -2po = 



2o (1-p) where p is the correlation coefficient. Now, the probability 
that a judge rates instance j as possessing more of the property than 



instance i may be expressed as Pr(Xj > x^) . We will operate on the 



inequality as follows: 



Pr(Xj > Xj^) = Pr (0 > Xj^ - x^) 



= Pr 



-(S. -S.) > (x. -X.) - (S. -S.) 

1 3 1 j 1 j 



= Pr 



S . S . 

3 - ^ 



(x. -X.) - (S. -S.) 

2 ^ 3 



(1-p) 



v^2a^(l - 



The right hand side of the final inequality above is a normally distr: 
buted random variable with a mean of zero and a variance of unity. Tl 



we have 
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Pr (x . > X . ) = Pr 
3 1 



S . - S . 
_J L 



> z 



v7^ 



(16) 



2o‘-(l - p) 



where z is the standard normal deviate. 

An estimate of the left-hand side of (16) may be obtained from the 
ranking information furnished by the judges. The proportion of judges 
who rank instance j as possessing more of the property than instance i 
may be used as an estimate of Pr(Xj > x^) . 

Let p^j be the proportion of judges who rate instance j as possess- 
ing more of the property to be scaled than instance i. Let z^^ be the 
value of the standard normal deviate (from the Normal Table) associated 

with p. that is, z. . is the value of z for which the leftward area 
1 ^ ^ 



under the normal N(z;0,l) curve is p. . 

J • 



Examples are 
P 



ID 



z . . 

ID 



0.92 

0.90 

0.50 

0.001 



1.645 

1.282 

0 

-3.0 



We now have, from Equation (16) estimating equations of the form 



z 



ij 




- 



p) ' 



(17) 



with one of these equations occurring for each instance pair i,j. In 
(17) , the left hand z^^ values come from the judges' rankings, being the 
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standard normal deviate associated with the proportion of judges who 
ranked instance j as possessing more of the property than instance i. 



On the right hand side of (17) , we« have S^-S^, the difference in two 



of the scale values we wish to obtain. 



Since we seek values of and on an interval scale whose uni* 



and origin are yet unspecified, we may use our freedom to specify unit 
and origin to simplify the mathematical development. Reserving speci- 
fication of the scale's origin until later, we can currently obtain a 
simpler form of (17) by specifying a unit for the scale such that 



(18: 



^^20^ (1-p) = 1. 

This is, of course, not an assumption. 

The scaling problem now stands as follows. We have n instances 



to be scaled; we seek values for S^, • 



, S . We have an n x n 
n 



array of values which came from judges' rankings, and we have a 

set of n{n-l)/2 estimates of the form 



z . . V S . -S . 
^ D 1 



(19) 



For n > 3, we will have more estimates than values to be estimat 
Viewing the various estimates as being equally useful, we will now 
propose a least-squares fit over the estimates (^19) . 

Before going ahead with the least-squares procedure, we must at 
this time pause to point out an important characteristic of the an 

If all judges rank, say, instance a as posessing more of the property 



than instance b, then we will have Pj^^ = 1.0, p^^ = 0, and thus z 



ab 



'ba 



and = - 00 . In practice, to avoid numerical bias by a small number 



43 



of judges, values corresponding to 



> 0.98 



and 



p . . < 0.02 

13 . 



are omitted from the array 



14 



Thus (if any) there will be an even 



number of "holes" in the z^^ array, symmetric about the diagonal. Our 



continued development must take into account the possibility that the 



array may be incomplete. 



Least-Squares Solution 



For the least-squares procedure, let <)>j be the set of rows possess- 



ing elements in the jth column of the z^^ array. Now ideally, z^^ = 



Sj thus for our least-squares fit, we will want to find a value 



for S . that minimizes 
3 



W.= y (z..-(S.-S.))^. 

3 iei. V 3 1 / • 



The algebraic expansion of the summation term is 



W . = 
3 



1 £ ( 



- 2^i3(Sj-S.) + (S.-S.)2 



ij. ^ " "i1 



yz..^-2s. y z.. + 2y z..s. + n.s.^-2s. y s. + ys.^, 
ieij ^ ie*. i£^. ^ ^ ^ ^ ic^,. " i4. ^ ' 
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where is the number of elements of For 

derivative and equate it to zero, viz.. 



a minimum, we take the 



3W. 



as . 

3 



- 2 I 



21 z..+2n.S.-2y S. 

i e ^ ^ ^ ^ ^ i c J) . ^ 



set 



- 0 , 



or 



n .S . - 
3 3 



le* j 



^ ^i = . ^ 



ie<|>j 



z . . 
13 



^ l,2,..*,ni 



( 20 ) 



Equation (20) represents a set of n linear equations in S,, S_, ..., S . 

12 n 

It may be shown that this set of equations is not independent (the read 

may wish to establish this by showing that the negative of any equation 

is equal to the sum of all remaining equations, or that the coefficient 

matrix is singular) . The set of equations becomes independent when any 

one equation is eliminated, leaving us with a set of n-1 equations in r 

unknowns. After arbitrarily setting a value for one of the (which i 

equivalent to setting the origin for the interval scale we are developi 

we can solve the set of equations using standard procedures such as 

17 

Gauss-Jordan elimination methods. 

Solving sets of simultaneous equations becomes a cumbersome 
activity when appropriate computing equipment is unavailable and when 
n is even modestly large. This is true even when the coefficient matr 
consists of small integers, as is our case. A method is given below t' 
reduce this effort by decreasing the number of equations which have to 
be fully solved. Savings in effort will occur only when there are col 
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in the array which are complete in that they have no missing elements 

To present our labor-saving approach, we return to our set of n 
equations of the form • 



n ;S . 
3 3 



ic (p . 
3 



S. = 
1 



ie<i>j 



ID 



= 1 , 2 , 



n . 



( 20 ) ' 



If there exists one or more complete columns in the z. array, we precede 
by establishing the origin for the scale by making the average scale value 
zero , i . e . , n 

. - 0 ' ( 21 ) 
n 



n 

which implies that J S = 0 . 

iii ^ 



Suppose that column k is complete, so that n =n. The equation for column 

n n ^ 

kis nS, - ys. = yz., 

i=l i=l 



n 

and immediately, y z^j^ (22) 




n 

since J S- = 0 . Thus, when a column is complete, we can immediately 

i=l 



compute the scale 
the average of the 
complete columns, 



value for the instance associated with that column as 

z. . in that column. After we have done this for all 
ID 

there will remain a set of incomplete columns associated 



with instances for which we do not yet have the scale values. All 
that remains is to (a) write out the equations of the form of (20) for 
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the incomplete columns, substitute the values of the s^ we have obtains 



from the complete columns, and (b) then solve this smaller, easier set 
of equations. 



In the procedure just described, column averages of complete z. 



columns are used as scale values. Thus if all columns are complete, tl 



entire set of scale values may be readily found by simply using coli 



averages. It must be emphasized that when elements are missing from a 
column, it is not correct to use that column average as nn estimate of 
scale value, even for an approximation. 



APPENDIX C. 



A NUMERICAL EXAMPLE 



Two hundred judges were asked to rank four types of equipment 
in terms of loss rate. They were instructed to record the item 
(A, B, C, and D) in rank order according to their best judgement, 
but to omit ranking any type of equipment with which they felt 
too unfamiliar to make a judgement about its loss rate. They 
were also instructed that if they felt that two items had the 
same loss rate, they should be recorded on the same line. 

A typical judge's response, then, would be a completed from 
as shown below.* 



Highest Loss Rate E 

A 

C 

Lowest Loss Rate q 

This input was tallied on the frequency array in the following manner. 
Since B was rated at the top of this judge's ranking, we go to the B 
column of the frequency array and in that column we record tallies in 
rows corresponding to instances (items) that this judge has ranked 



*Note the cues given to the first and last rankings, which help reduce 
the chance that a judge will list instances in reverse order. 
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below B.: 



f . . 
ID 



A 

B 

C 

D 



Then we move to the judge's second choice, find its column, and recoi 
tallies in rows corresponding to instances ranked below the second ci 






A 

B 

C 

D 



A 



D 



1 1 
1 1 



Continuing to the judge's third ranked instance, C we obtain 



f . . 

ID 



A 

B 

C 

D 



B 



D 



1 1 
111 



as this judge's contribution to the total tally for all judges 
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The completed frequency array for all 200 judges is shown below. 



f. . 


A 


B 


C 


D 


A 




33 


1 


45 


B 


167 




30 

« 


118 


C 


195 


166 




172 


D 


155 


82 


24 





Adding across the diagonal, we see that not all 200 judges made all 
comparisons. This was due to the fact that several judges omitted 
them C from their rankings. 

Continuing, we next must compute the p. . array from the f. . array 
by 



f. . 



ij 



f . .+ f . . 
ID Di 



Because not all judges ranked all items , the divisor in some cases 
will be less than 200. The p^^ array is computed as 



sums 



Pij 


A 


B 


C 


D 


A 


0.5 


0.165 


0.005 


0.225 


B 


0.835 


0.5 


0.153 


0.590 


C 


0.995 


0.847 


0.5 


0.878 


D 


0.775 


0.410 


0.122 


0 . 5 


s 


2.605 


1.420 


0.280 


1.693 



From the p^^ array we see that in comparing C and A the judges were too 
close to unanimity, and thus we will omit this comparison < 0.02, 



p_, > 0.98) in the z. . array. From a table of the normal distribution, 
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the array is 




Since the z^^ array has cells which are empty, we cannot use t 



column averages to compute scale values for all the items. Instead, w 
have to solve simultaneous equations to obtain a least-square solution. 
Two columns of the z^^ array are complete, and two are not. We begin 



by computing the column sums: 



Iz • 



ic4>] 

n 



j 


A 


B 


C 


D 


j 


1.729 


-0.177 


-2.194 


0.642 




3 


4 


3 


4 



Implicitly setting the origin of the scale by taking the average scale 
value as zero, we immediately compute column averages for the two 
complete columns and obtain two scale values: 



S = = -0.044 

o 



and 



"d = 



4 

0.642 



= 0.161 



For the remaining columns (A and C) , the set of equations of tl 
form of (5) is 



3^A - ^ + S 3 ) = 1.729 






-2.194. 



Since 
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we may write the two equations as 



/ 



2S^ = 1.729 + 0.117 
« 

= 1.846 

and 



and thus we have 



and 



2S^ = -2.194 + 0.117 
= -2.077 



= 0.923 



= -1.039 . 



These values are still with unspecified unit and origin. If 
we estimate that the loss rate for C is 10, and the loss rate for 
item D is 30, then Equation 15 becomes 

Lj = 27.32 + 16.67S^ , 

and we have, finally. 

Item Loss Rate 
A 42.7 

B 25.6 

C 10.0 

D 30.0 



APPENDIX D 



RANKINGS BY JUDGES 



Judgs: 


1 


2 


3 


' 4 


5 


6 


7 


8 


9 


10 


Highest 


46 


40 


40 


43 


41 




13 


13 


20 


46 


Chanca 
of Loss 


41 


41 


41 


10 


40 




14 


14 


23 


40 




42 


42 


42 


16 


42 


40 


49 


49 


26 


41 




40 


43 


19 


20 


43 


41 


43 


40 


29 


42 




10 


49 


14 


19 


13 


42 


41 


42 


23 


13 




16 


13 


13^ 


_ 14 


30 


30 


40 


43 




49 




43 


14 


30 


17 


35 


46 


42 


41 




43 




49 


16 


10 


46 


49 


10 




17 




14 




13 


30 I 


26 


42 


46 


20 


46 


11 




17 




30 


17 


^ 43 


41 


29 


43 


30 


16 




19 




35 


23 


49 


40 


23 


29 


19 


30 




30 




14 


35 


16 




10 


16 


10 


35 




23 




17 


29 


17 




17 


17 


29 


10 




26 




23 


26 


20 


43 


14 


19 


23 


19 




35 




26 


20 


21 


49 


19 


14 


50 


23 




16 




19 


19 


29 


30 


20 


13 


26 


20 




20 




20 


46 


23 


35 


26 


21 


11 


26 




21 




29 


21 


35 


29 


21 


23 


20 


21 








21 


10 




26 




26 


35 


29 








50 






23 




35 


16 


46 






Lowast' 
Chance 
of Loss 








21 




49 


21 


50 







Daiets: 








43 








23 




Drr.its : 


11 


11 


11 


11 


11 


11 


17 


many 


11 






50 


46 


13 


16 


50 




40 








50 


50 


50 











52 



53 



Judge: 


n 


lA 


2 ^ 


M 


2 ^ 


16 




ii 


Highest 


41 


41 


41 


13 


14 


42 


16,10 


40,43 


Chance 
of Loss 


40 


40 


11 


14 


17 


, 41 


41 


42 




42 


42 


10 


49 


13 


43 


42 


30 




23 


23 


16 


43 


41 


14 


49 


46,49 




30 


30 


40 


40 


42 


10 


40 


13 




35 


35 


42 


17 


30 


13 


43 


14 




29 


29 


43 


42 


40 


30 


13 


16 




19 


19 


30 


19 


43 


35 


14 


20 




10 


10 


35 


10 


20 


40 


17 


10 




11 


11 


26 


16 


35 


17 


30 


19 




26 


26 


49 


46 


10 


29 


35 


29 


1 


13 


13 


23 


11 


29 


46 


19 




1 

1 


14 


14 


46 


30 


26 


49 


23 




i 


16 


16 


20 


29 


21 


13 


29 






17 


17 


29 


35 


49 


20 


26 






20 


20 


17 


23 


46 


19 


20 






21 


21 


19 


26 


16 


21 


21 


35 








13 


20 


23 


23 


46 






46 


46 


14 


21 


19 


26 








43 


43 


21 


50 


11 


50 






Lowest 
Chance 
of Loss 


49 


49 


50 




50 






26,21 








Delete: 












13 






Omits : 


50 


50 




41 




11 


11 


11 














16 


50 


17 



23 

41 

50 
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Judge: 


19 


20 


21 


22 


Highest 


13,14 


13 


13 


49 


Chance 


43,49 


49 , 17‘,14 


43 


13,14 


of Loss 












17 


40 


49 


17 




16,19 


41 


14 . 


20 , 19 , 16,21 




40 , 41,42 


10,11 


40 , 41,42 


40 




29,30 


43 


10 


43 




10,11 


16 


16 


42 




20,26 


42 


17 


41 




21 


30 , 19,35 




29 




23 


26 , 23,29 


30 


10,23 




46 


20,46 


35 


26,46 




35 


21 




30,35 




50 




29 










26 










19 





Lowest 




Chance 




of Loss 


20 



Delete: 
Omits : 



50 



11 

46 

50 



11 

50 
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Judge : 

Highest 
Chance 
of Loss 



23 

13.19 
49 

40 , 41 , 42,43 

30,35 

10 

14,17 

46 

16.20 
23 

11,29 

26 

21,50 
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